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Abstract 

In this paper we show how string rewriting methods can be applied to give a new 
method of computing double cosets. Previous methods for double cosets were enu- 
merative and thus restricted to finite examples. Our rewriting methods do not suffer 
this restriction and we present some examples of infinite double coset systems which 
can now easily be solved using our approach. Even when both enumerative and 
rewriting techniques are present, our rewriting methods will be competitive be- 
cause they i) do not require the preliminary calculation of cosets; and ii) as with 
single coset problems, there are many examples for which rewriting is more effective 
than enumeration. 

Automata provide the means for identifying expressions for normal forms in infi- 
nite situations and we show how they may be constructed in this setting. Further, 
related results on logged string rewriting for monoid presentations are exploited 
to show how witnesses for the computations can be provided and how information 
about the subgroups and the relations between them can be extracted. Finally, we 
discuss how the double coset problem is a special case of the problem of computing 
induced actions of categories which demonstrates that our rewriting methods are 
applicable to a much wider class of problems than just the double coset problem. 
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1 Introduction 



Given a group G and two subgroups H and K, the double cosets are the equiva- 
lence classes of the relation ~ where g ~ g' -v=> hgk = g' for some /iG H, k G K. 
The set of double cosets is written H\G/K. Combinatorially, the double cosets 
are the orbits of the left action of H on the right cosets G/K, and also the orbits 
of the right action of K on the left cosets H\G. Double coset techniques give 
examples of deep and wide applications of group theoretic methods in chem- 
istry and physics (Ruch and Klein, 1983). For example, there are applications 
through Polya's theory of counting, to considerations of deuterons colliding 
in scattering theory (Pletsch, 2001). Real semisimple symmetric spaces are 
often characterised by a pair of commuting involutions of a reductive group 
and many of their properties are studied in this setting - in this case double 
cosets are of importance for representation theory of p-adic symmetric K- 
varieties (Helmink and Brion, 2000). Double coset computation can be seen 
as a way of constructing finite quotients of HNN-extensions of known groups 
or as a way of constructing groups given by symmetric presentations (Curtis, 
1992). 

There are a number of different questions which arise if we want to compute 
with double cosets, for example: the enumeration of the double cosets; finding 
a set of representatives for them; deciding questions such as whether a pair of 
group elements lie within the same double coset or not; and proving either case. 
There are number of algorithms for computing such double coset problems 
but these are regarded as incomplete. Indeed, in Section 4.6.8 of the recently 
published survey book on computational group theory (Holt et al., 2005) we 
find "Unfortunately, no really satisfactory algorithm for solving this problem 
has been found to date." In 1981 the first algorithmic methods for computing 
double cosets were published (Butler, 1981), and applied to groups of order 
^ 10 4 . The approach was a variation on Dimino's algorithm (Butler, 1991) 
for computing a list of elements of a small group. In (Holt et al., 2005), the 
basic approach for permutation groups is to use orbit methods to compute 
left or right cosets, and then orbits of these cosets to obtain double cosets. 
As pointed out in (Holt et al., 2005), this may involve the calculation of 
a large number of cosets in order to determine a small number of double 
cosets. More recent methods for computing double cosets of finitely presented 
groups use Todd-Coxeter procedures (Linton, 1991). All of these methods have 
been implemented in the commonly used programs for computational discrete 
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algebra, GAP (GAP, 2004) and MAGMA (MAGMA, 2005). 

The primary alternative to Todd-Coxeter procedures for ordinary coset enu- 
meration and computation of groups given by presentations is string rewrit- 
ing (Brown and Heyworth, 2000; Sims, 1994). In finite settings the two ap- 
proaches are comparable: certain problems being more effectively addressed 
by the enumerative method and others benefiting more from a rewriting ap- 
proach. However, for cases involving infinitely many elements, rewriting rather 
than enumeration is the natural choice. 

This paper demonstrates how string rewriting can be applied to the problems 
of computing double cosets, giving a new alternative to the Todd-Coxeter 
methods and one which can further be applied to infinite groups. In particular, 
this paper makes the following contributions. 

• The introduction of the notion of a double coset rewriting system and asso- 
ciated Knuth-Bendix completion algorithm as a mechanism for attempting 
to decide whether two elements of a group lie in the same double coset. 

• The specification, in Section 4, of a process which takes a finite complete 
double coset rewriting system and constructs a finite state automaton whose 
language is a set of unique normal forms for the double cosets. 

• The specification of a higher dimensional version of the Knuth-Bendix al- 
gorithm and logged double coset rewriting. This gives, for example, presen- 
tations for the subgroups defining the double cosets. 

• A discussion of the implementation of these algorithms as a deposited pack- 
age kan for GAP. 

• In Section 7 we put our algorithm in context by showing how it arises as a 
special case of rewriting for an induced action of categories, using a category 
H -e> K constructed from the two subgroups. 

The authors would like to thank Steve Linton for helpful discussions, especially 
regarding the GAP implementations of the methods presented in this paper; 
and Tim Porter for help with applications of category theory. Thanks are also 
due to the referee for several helpful suggestions. Figures were typeset using 
XFig. 



2 Rewriting for Double Cosets 

If we consider using rewriting to solve double coset problems we have a choice: 
to develop a specialised type of rewriting for this situation, or to rephrase the 
problem in a way that allows existing techniques to be applied. The former 
method has the advantage of specialty - the "double coset rewriting systems" 
can be examined in isolation as though they were in some way an advance on 
existing methods rather than a useful application. This is certainly of some 
value if one is wishing to write a very specialised program, designed to com- 
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pute only double coset problems and investigate the particular properties of 
rewriting systems of this type, but it can obscure the simplicity and the best 
features of the result. Therefore, we choose to adopt the most straightforward 
method - simulating the required computations by embedding the group G in 
a particular free monoid and then applying standard procedures (Book and 
Otto, 1993). We then have to show that the structure we wish to compute 
coincides with the rewriting model used. 

Definition 2.1 (Presentation of a Double Coset System) 

Let G be a group with monoid presentation mon(Xc, Rq) and let 9 : Xq — > G 
be the natural monoid homomorphism. Let Xy\,X^ C Xq be such that Yh = 
9(Xh) andY^ = 9(Xk) are sets of generators for the subgroups H and K respec- 
tively. Then we shall say that the quadruple (Xq, Rq, X h , X k ) is a presentation 
of the system of double cosets H\G/K. 

If R generates an equivalence relation or congruence on a free monoid S then 
the class of s G S is denoted [s]r. Similarly, we write H\G/K = {[g]^ \ g G G}, 
where ~ is the relation defined at the beginning of Section 1. If mon(Xc, -Rg) 
is a monoid presentation for G then the free monoid T + in which we com- 
pute is generated by Xq together with two extra (tag) variables H and K. A 
string HwK represents the double coset and we require a congruence 

such that HwK Hw'K if and only if 9w' ~ 9w. Clearly T + contains 
many elements that are not of the form HwK, but these do not arise in the 
computations performed when completing the rewriting system which deter- 
mines the double cosets. This is the key observation which allows us to use 
standard methods. 

Theorem 1 (Double Coset Rewriting) 

Let (Xq, Rq, Xh, X^) be the data for the double coset system H\G/K, let H 
and K be symbols, and let T be the subset of terms of the form HwK of the 
free monoid T + = ({H, K} U Xq)* where w G Xq. Define 

R = R C U {(Hh, H):heX H }U {(kK, K) : k G X K }. 

Let be the reduction relation generated by R on the free monoid T + , and 
let A R be the reflexive, symmetric, transitive closure of -^r, which is the 
congruence generated by R. Then there is a bijection of sets 



Proof We first show that there is a well-defined map 
T G 

4> : — > — where ^>([HwK]r) = [9w]^ . 

^R ~ 

If [HwK] R = [Hw'K] R , there exists a sequence w = W\,w 2 - ■ ■ ,w n = w' in 
Xq such that for i — 1, . . . , n— 1 either (wi, Wi + \) or (wi + %, Wi) has one of the 
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following forms: 

i) (ulv,urv) for some (l,r) G -Rg, u ,v G Xq, 

ii) (hv, v) for some h G Xh, v G Xq, 

iii) (life, u) for some fc G X«, u G X^. 

Since 9 is a monoid homomorphism, in the first case 9{vjj) = 9(wi + i), in the 
second case 9(hv) ~ 9{v) and in the third case 9{uk) ~ 9{u). Thus in all cases 
[9(wi)]^ = [9(wi+i)]~ as required. 

Secondly, let r : G — > Xg be a section of 9 so that {9or)g = g and (ro9)g <^r g g 
for all g G G. Further, r(gig2) ^r g t(5 , i)t(5 i 2) since maps both t(<7i)t(<72) 
and r(f/ip 2 ) to #i# 2 - Define 

0' : - - -1- where (f>'([g}~) = [H(rg)K] R . 

To verify that 0' is well-defined, suppose [g]^ = \g']~ for some g,^' G G. Then, 
by the definition of ~, we have h G X^ and k G X^ such that g' = (9h)g(9k), 
so that 

H(rg')K A R H(r9h)(rg)(r9k)K ^ Hh(rg)kK ^ R H(rg)K. 

Finally, we observe that <f>(<f/ (\g]~)) = <p{\H(rg)K)} R ) = [9(rg)]^ = \g]„, and 
that <P(<f>([HwK\ R )) = <f>([(9w)]„) = [H{r{9w))K} R = [HwK] R since R G C i2. 
Thus is a bijection with inverse 0'. □ 

Note the use of the tags H and K. They provide a particularly simple way of 
deleting elements of Xh from a word providing they occur at the far left of 
the word and similarly for deleting elements of Xk providing they occur at the 
far right of the word. Given the double coset presentation (Xg, -Rg, Xh, Xk) 
as in the theorem above, we may refer to R as a double coset rewriting system 
for H\G/K. Of course R may not be complete (confluent and noetherian) and 
so the natural next step would be to use Knuth-Bendix completion to try 
and obtain an equivalent, but complete, rewriting system. As justified above, 
we choose to perform this completion by considering rewriting over the free 
monoid T + rather than the set T. However, we must then be sure that if — > R i 
is complete on T + , and its closure ^ R i coincides with A fl , then the restriction 
of —> R i to T is also complete. Fortunately, this is obviously true as — > R is closed 
on the subset T, that is, if wi —> R w 2 then wi G T if and only if w 2 G T. To 
see this, note that R never removes or adds tags. Thus, we can immediately 
state the following corollaries. 

Corollary 2 If the double coset rewriting system R can be completed on T + 
then we have a solution to the problem of deciding whether two elements g,g' 
of G lie within the same double coset. 

Corollary 3 If the double coset rewriting system R can be completed on T + 
then we can find a unique normal form for each double coset. 
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Remark 4 (Implementations) 

Besides the fact that the string rewriting methods we have presented enable 
us to tackle problems involving infinite groups, they also allow us to imme- 
diately use existing string rewriting programs such as those in GAP and in 
KBMAG (Holt, 1996) to compute double cosets. 

An alternative approach is to complete a rewriting system for G and then 
construct a double coset rewriting system on the elements of G. Such a system 
obeys more subtle laws than a standard rewriting system on a free monoid. For 
example, if g > g' in the termination order used by the completion process, 
we may not deduce that gg~ l is greater than g ' g~ A . Consequently, we believe 
the approach we have chosen is cleaner than if we were to have worked with 
multiple rewriting systems at different levels to describe the one structure. 

While it is straightforward to try to use a standard Knuth-Bendix completion 
program to calculate the completion of a double coset rewriting relation on the 
free monoid T + , it will necessarily be less efficient than a specialised Knuth- 
Bendix program which, for example, restricts itself to the non-free monoid T. 
For example there will be tests for more overlaps between words wK and zK 
than can possibly arise: we know (but the program does not) that the only way 
in which an overlap can occur is when z = uw or w = uz, since the tag symbol 
K will not occur within the strings w or z. A specialised program would also 
allow different types of ordering, treating symbols if, A' in a way different 
from those in Xq, yielding results that could not be obtained with a standard 
ordering. If one wishes to do many calculations of this type, it would be worth 
refining the system to recognise tags and deal with tagged rules sensibly, and 
to separate rules into subsystems which are completed separately. Such an 
approach would not be designed specifically for double cosets and could have 
many other applications (Brown and Heyworth, 2000). 



3 Completion of Rewriting Systems for Double Cosets 

As we have seen, if the double coset rewriting system R is complete, we can 
solve problems such as whether two elements of the group belong to the same 
double coset. Usually, R is incomplete, and so we attempt to convert it to an 
equivalent complete system. We can apply the Knuth-Bendix completion pro- 
cedure (Knuth and Bendix, 1970) to R on T + in the standard way, as detailed 
below. If R completes on T + , we are required to prove that the restriction of 
to T is preserved throughout the algorithm. 

Algorithm 1 (Completion) 

Kl (Input) Start with a set of pairs R = Rq U {(Hh,H) : h G Xh} U 
{(kK, K) : k G X«} C T + and a compatible well-ordering on T + . 
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K2 (Search) Find all overlaps: pairs of rules (h,ri), (h,^) which may be 
applied to the same word and which coincide on some subword. There are 
essentially two cases, ul\V = l 2 or ul\ = l 2 v for some u, v £ T. Add each 
pair (uriv,r 2 ) or (wi,r 2 t>) to a list of critical pairs. 

K3 (Resolve) Attempt to resolve each critical pair by reducing both terms 
with respect to the rules in R. If the reduced terms are equal then the 
pair has resolved, otherwise the reduced pair is orientated according to the 
well-ordering and added to a set of new rules and to R. 

K4 (Loop) If no new rules were added then go to the next step. Otherwise, 
repeat the last two steps with the new set of rules, checking pairs that arise 
between the new rules and between the new rules and the old rules, but not 
pairs just between old rules (as these have already been checked). 

K5 (Output) Return the resulting R c , a complete rewriting system on T + 
with respect to the given well- ordering. 

We now prove that if the input for the algorithm is a double coset rewriting 
system of the form specified earlier, then the algorithm will attempt to produce 
an equivalent complete system. 

Theorem 5 (Completeness of Double Coset Rewriting Systems) 

Let the input for the above algorithm be a double coset rewriting system R as 
given in Theorem 1, such that the algorithm terminates, giving output R . 
Then the restriction of -^rc to T is a complete rewriting system equivalent to 
the restriction of — >r to T . 

Proof We prove the result directly, by showing that no step in the completion 
procedure alters the restriction of to T. The argument holds because of 
the form of the input and the way in which new pairs are generated. 

It is convenient (at each stage of the algorithm) to partition a rewrite system 
R into four subsets Rh, R^, Rq and -Rhk, depending on whether the rule 
involves H, K, neither or both. The subset R\-\k is initially empty. Formally: 



We observe that in step K3 of the algorithm a new rule is generated from an 
overlap of the left hand sides of two existing rules (followed by their subse- 
quent reductions). 

Overlaps of rules (h,ri) and (h,^) in Rg m ay be separated into five types. 
In Table 1, l\ is either a prefix of l 2 ] a suffix of 1%; an internal subword of I2', 
or the overlap is offset to the left or the right. Neither of u, v may equal the 
empty word id. 




{(l,r)eR\l,reX* G }, 
{(HI, Hr) ER\l,re X* G }, 
{(IK, rK) eR\l,re X G }, 
{(HIK, HrK) e R\l,r e X£}. 
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type 


overlap case 


new rule to add 


picture 


prefix 


7 „, 7 


(riv, r2) 




suffix 


Ul\ — 12 


(urx,r 2 ) 


_ 


internal subword 


ul\v = I2 


(uriv,r 2 ) 




left offset 


l\V = 11I2 


(riv,ur 2 ) 




right offset 


ul\ = I2V 


(uri,r 2 v) 





Table 1 : Five types of overlap in G. 

Using the same terminology for the other types of rule, we see that the overlaps 
which may occur are of the types shown in Table 2. 



overlap words 


prefix 


suffix 


internal 


left offset 


right offset 


h,h 


/ 


/ 


/ 


/ 


/ 


h,Hl2 




/ 


/ 




/ 


h,l 2 K 


/ 




/ 


/ 




Hl\, HI2 


/ 










Hh,l 2 K 








/ 




hK, l 2 K 




/ 








h,Hl 2 K 






/ 






Hh,Hl 2 K 


/ 










hK, Hl 2 K 




/ 









Table 2 : Overlaps types for all pairs of rules. 



In each case we observe that the new pair added to R will not change the 
definition of A R when it is restricted to T since the new pair is an element of 
A fl and also an element of T x T. 

The reduction of the new rules with respect to the existing rules also gives 
a pair which is already an element of A fl when it is restricted to T. This is 
because replacement of a substring in a word in T + by will preserve the 
positions of the tags. □ 



4 Automata for Double Coset Rewriting Systems 

When the number of double cosets is infinite, it is not possible to list all their 
normal forms, so a regular expression giving an impression of the shape or 
pattern of these forms may be useful. In simpler string rewriting systems, 
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for example with left or right cosets, we can build up a catalogue of the 
normal forms. In finite cases where we have a length non-increasing order 
this is extremely effective, and even in infinite cases it can serve to show up 
patterns. The cataloguing procedure relies heavily on the fact that, if a term 
is reducible and generators are appended onto a chosen end, then the term 
remains reducible. 

In the case of double cosets and doubly tagged strings we cannot use these 
methods. If a term HIK is reducible, then one of I — > r or HI — > Hr or 
IK — > rK or HIK — ► HrK is true, but we cannot assume that either HlxK 
or HxlK are reducible. For example, if HI — > Hr, then we are unable to 
deduce that Hxl — > Hxr. Since we cannot use cataloguing, we turn to the 
alternative: in string rewriting we use automata when we wish to find an 
expression for the set of normal forms and cataloguing is insufficient. Here, we 
describe techniques for constructing automata whose languages are the sets 
of normal forms for our double cosets. First we recall the construction of an 
automaton for accepting normal forms in G. 

We will use the following notation. For any set of rules R we set \(R) = 
{/ | (l,r) G R}, the set of left-hand sides of these rules. Then p\(R) is the set 
of all prefixes of the rules and ppl(-R) is the set of all proper prefixes: 

pl(-R) = {u | (uv,r) G R, u 7^ id}, ppl(-R) = {u | (uv,r) G R, u,v ^ id}. 

Similarly, s\(R) and psl(i?) denote the sets of suffices and proper suffices. A 
non-deterministic automaton N, with state set S; alphabet E; initial state 
s G S; transition function 5 : S x £ — > 2 s ; and accepting states A C S, is 
written N = (S, E, sq, 5, A). A deterministic automaton has iiSxE^S. 

Definition 4.1 (Word acceptor for G) 

The word acceptor of a group G = mon(Xc, Rq) with a finite complete rewrite 
system Rq is constructed as follows. First form a non-deterministic automaton 

N G = ( ppl(iig) U {id, sink}, X G , id, S G , {sink}) 

whose states consist of all proper prefixes p from Rq; the identity word as 
initial state; and a sink state sink which is the only accepting state. 

The transition function is given by 

<5c(sink, x) = {sink} for all x G Xq, 

{{sink} if px = ul for some I G I(-Rg), else 
{Pi G ppl(-Rc) I PX = for some u { G Xq }. 

Standard results of automata theory (Cohen, 1991; Lawson, 2004) allow us to 
determinize N.g (using the accessible subset construction); take the comple- 
ment (accepting states become non-accepting, and conversely); and minimize, 
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giving a deterministic automaton A G which accepts only the normal forms of 
elements of G. 

Theorem 6 (Word acceptor for H\G/K) 

Let R c be a finite, complete, double coset rewriting system for subgroups H 
generated by Yh = 0Xy\ and K generated by Y"« = @X^ of the group G which is 
given by the monoid presentation mon(Xc, Rq) . Let T and T + be defined as in 
Theorem 1. Then there is a regular expression representing a regular language 
L over T + such that L = {[g]~ : g G G}. 

Proof We define a non-deterministic automaton N with input alphabet X = 
Xq U {H, K} which accepts exactly the set irr R c(T) of irreducible elements of 
T with respect to — >#c. 

As before, we partition the rules in R c into Rq U R^ U R^ U R^k, where Rq is 
the complete rewrite system for G. The automaton has four main components: 

• the non-deterministic form J\[ G of the word acceptor for G, with states 
as in Definition 4.1 ; 

• an H-tree, whose states are S*h = {LL\d} U ppl (i?^ ) ! 

• a K-tree, whose states are 5k = {id-ft'} U ps\(R^) ; 

• an H K-tree, whose states are Shk = {H\d-K} U (pp\(Ry\ k )-K) . 




Figure 1 : Sketch of a double coset automaton. 



There are two additional states, an initial state in it and a normal form state 
norm which is the only non-accepting state. An informal sketch showing how 
these components and states are connected by transitions is shown Figure 1. 

The formal definition of the non-deterministic automaton N. is 
N = ( S, S = X c U {H,K}, init, 5, S\ {norm}), where 
S = {init, norm} US G US H US K U Shk- 

The transition function 5 is defined in Table 3, where s e S, x G Xq, a G 
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E, p G pp\(R%), Hp G ppl(itf), qK G psl(i^) and Hp-K G ppl^)"^- 



1 ' ^ * ■ • 1 ( 1 1 l> 1 

lUCclLlUIl 


transition 


from in it 


<5(init,#) 


= {\d,H\d,H\d-K} 






(5(init, a) 


= { 


sink} when a ^ H 




by H 


5{s,H) 


= { 


sink} when s ^ in it 




by K 


6(p,K) 


= { 


norm} 






5(\dK,K) 


= { 


sink} 






6(Hp-K,K) 


= { 


sink} if HpK G \{Rhk) 




H-tvee 


5(Hp, x) 


= 1 


{Hpx} if Hpx G ppI(-Rh) 
{sink} if Hpx G I(-Rh) 


K-tvee 


5(xqK, x) 


= {qK} if xqK G psl(i?K) 




HK-tree 


5(Hp-K, x) 


= {Hpx-K} if Hpx G ppl(i?HK) 




in N G 


5{p, x) 


-1 


{sink} if px = ul for some / E l(i?G 
k {Pi S ppl(i?G) pa; = UipJ U {xqK 


), else 

xgi^G \(R K )} 


otherwise 


S(s, a) 


= 







Table 3 : Transition function 5 for double cosets automaton. 



The extended state transition function 5* : S x S* -> 2 5 is such that, for 
i G T, the intersection of <5*(init, t) with 5* \ {norm} is non-empty if and only 
if t is a word in S* which is not in T or is reducible. 

Just as we converted N.g to A G , we make N. deterministic; take its complement; 
and minimize. The language L recognised by the resulting automaton A is 
T + — (T + — irr R c(T)) = irr fi c(T). Hence, by Kleene's Theorem, L is regular. 
Since R c is a complete rewriting system on T, there exists a unique irreducible 
word in each class of T with respect to Therefore the set \rr R c(T) is 
bijective with T/ <^r= L. 

The automaton A gives rise to a system of right linear language equations with 
a unique solution, which is a regular expression for the language L accepted by 
the automaton. The regular expression can be obtained by applying Arden's 
Theorem to solve the language equations. □ 

Thus an automaton A can be constructed from the complete double coset 
rewriting system and a regular expression for the set of double cosets L is ob- 
tained from solving the language equations of the determinized and minimized 
complement of A. Section 6 includes some examples of these automata. 
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5 Logged Double Coset Rewriting 

It is often useful to label the original rewrite rules and record how they are used 
during Knuth-Bendix completion, and then during the reduction of words. One 
instance is when we require precise proof of a particular equivalence in terms 
of the original data. 

Suppose that a s : s — > t is a rewrite rule. Then we know that the rewrite 
usv — > utv can be performed, and a reasonable label for this is ua s v. Similarly 
if at : t — > q then we may perform a s followed by at, rewriting s — > q, which 
we choose to label a s » at- Thus any sequence of rewrites may be recorded by 
a combination of labels of the form: 

uia^i • u 2 a S2 v 2 • u n a Sn v n . 

The algebra of recorded rewrites is formalised by observing the sesquicategor- 
ical or 2-categorical structure. Briefly, a 2-category consists of 0-cells (objects 
•i), 1-cells (arrows between objects (wij : •» — > and 2-cells (arrows be- 
tween arrows (a : =>- w'^)), with a category structure on the 1-cells (arrow 
composition) and two compatible (by the interchange law) category structures 
(horizontal and vertical composition) on the 2-cells, which preserve sources and 
targets (Mac Lane, 1998). 

In particular logged double coset rewriting, is formalised in terms of a 2- 
category whose 0-cells (vertices) and 1-cells (paths along arrows) are illus- 
trated in the following graph. 

G 

o 

* H *" * K *" * 

The generating 2-cells are 

a^ '■ Hhi — > H, for each hi e X^, 
ocii : h —> n, for each (k, n) G Rg, 
and a^ : kiK — > K, for each fcj G X^. 

The vertical composition of 2-cells, written a«7, is the composition of the two 
rewrites when the source of the second coincides with the target of the first. 
We may also "whisker" the 2-cells with suitable 1-cells. For example, the 2-cell 
ai t may be whiskered by Hu on the left and v on the right to obtain a 2-cell 
(Huai.v : HuliV — > Huriv). Finally, we require the interchange law, so that 
it does not matter in which order we combine a pair of 2-cells which rewrite 
non-overlapping parts of a string. This defines the horizontal composition, 

ao/3 := a src((3) • tgt(a) (3 = src(a) (3 • a tgt((3) , 
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and corresponds to the fact that if rewrite rules do not overlap on a string then 
it does not matter which one we apply first. Note that whiskering is equivalent 
to composing with identity 2-cells, uav = 1„ o a o 1„ where (1„ : u =>- u) for 
u G S, and that av o (3 = a o v(3. 

Logged rewriting for monoid presentations is explained in detail in (Heyworth 
and Johnson, 2005) and we shall only recall the key ideas here. 

The essential difference between the logged version and the standard Knuth- 
Bendix algorithm is the level of detail it records. If we have an overlap which 
introduces a new rule, we require an expression for the new rule in terms of 
the original labels. Often we will be in the situation where w reduces to w\ 
by one sequence f3\ of 2-cells and to w 2 by another sequence /3 2 - Assuming w 2 
is the larger string, we add in the rule w 2 W\, and note that this relation 
can be achieved by "un- reducing" w 2 to w and then reducing w to Wi. This 
"un-reducing" is more formally known as an inverse derivation and gives the 
vertical composition of 2-cells a groupoid structure. In this situation we add 
the 2-cell fi 2 x • Pi at the same time as the rule w 2 — > w\. 

Assume that, using these methods, we obtain a complete, logged, rewrite sys- 
tem for the double cosets, which means that we have a 2-cell associated with 
each of the rules. Suppose now that we have two group elements g\ and g 2 , 
represented by strings w\ and w 2 in the free monoid, so that 9{w\) = g\ 
and 6(w 2 ) = g 2 . We can determine whether or not g\ and g 2 lie within the 
same double coset by rewriting Hw\K and Hw 2 K. If they both reduce to 
the same string HzK, then we can examine the logs of the reductions to find 
hi, . . . , h m G Ah and ki, . . . ,k n G Ak such that 

6{h?) ■ ■ ■ 6{h£) gi e(ki^) ■ ■ ■ 0(k%r) = g 2 . (1) 

The following section includes examples of this computation. 

In the case of left or right cosets, the logs of the complete rewriting system may 
be used to derive a presentation for the subgroup itself. In other situations 
the logs and particularly the logs of circular rewrites (endorewrites) have more 
interesting interpretations and applications (Heyworth and Wensley, 2003). In 
the double coset case we can make the following observations. We use E{w) 
to denote the set of endorewrites of the string w, the set of 2-cells associated 
to rewrites of w back to itself. 

i) The sets E(Hw) are bijective with each other for all w in A*. These give 
information about the group H in the form of a presentation (Ghani and 
Heyworth, 2003). 

ii) Similarly, the sets E(wK) give a presentation of K. 

iii) The sets E(w) are all bijective, and these give generators for the module 
of identities among relations for the group G (Heyworth and Johnson, 
2005; Heyworth and Wensley, 2003). 



13 



iv) The sets E(HwK) are not all bijective in general. However, in the case 
that 9{w\) = 0(w 2 ), there is a bijection between the sets E(Hw\K) and 
E(Hw2K). In general each endorewrite of this type gives us informa- 
tion regarding the relationship between the subgroups H and K within G. 
Generators for the subgroup H fl K can certainly be obtained in this way. 

6 Examples 

The examples given below were calculated using a prototype package kan, 
available from (Heyworth and Wensley, 2005). This is a collection of GAP 
functions which are designed to tackle rewriting problems by translating them 
to a categorical framework; using a generalised Knuth-Bendix type algorithm 
to solve the problem; and then translating back into the format appropriate 
for the structure in question. The double coset functions in kan make use of 
functions from the GAP package automata (Delgado et al., 2005). The main 
purpose of these examples is to demonstrate the methods we have presented 
and the fact that they can be applied to a wider class of problems than could 
previously be computed, rather than to be technically impressive. 

Example 7 (A finite double coset rewriting system) 

Let G be the free group on generators {a,b} and let H = (a 6 ), K = (a 4 ). 
(Varying the powers of a gives a family of examples of this type.) The double 
coset HK contains a 2 = a gcd ^ 6,4 \ and it is clear that one set of double coset 
representatives is 

{ HK, HaK, Ha%a j K, H^bub^K | i < 5, < j sC 3, u £ {a, b}* }. 

The initial set of rules is 

{(Aa,\d), (aA,\d), (56, id), (bB,\d),(Ha 6 ,H), (a^K, K) }. 

After completion, the last two rules are replaced by 

{(ifa 4 , HA 2 ), (HA 3 , Ha 3 ), (a 3 K, AK), (A 2 K, a 2 K), (Ha 2 K, HK), (HAK, HaK)}. 

Note that two HK-rules appear, reflecting the fact that H V K = (a 2 ). The 
non-deterministic automaton N has 22 states, 

{init, norm, sink} U {\d,a,A,b,B} U {H, Ha, Ha 2 , Ha 3 , HA, HA 2 } 
U {K, aK, a 2 K, AK} U {H -K, Ha-K, Ha 2 ■ K, HA-K}. 

Determinizing N. gives an automaton with 24 states which, after complemen- 
tation and minimization, reduces to a deterministic automaton with 15 states 
and transitions shown in Table 4 (where N, S, I correspond to norm, sink, init). 
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Table 4 : Minimal double coset automaton for Example 7. 
Example 8 (The Trefoil Group) 

This is an example in which the group has a finite rewriting system, but the 
double coset system is infinite. Starting with an initial monoid presentation 
with rules 



[ a x = (Xx, id), a 2 = {xX, id), a 3 = (Yy, id), a A = (yY, id), a b = (x 3 , y 2 ) ], 

the fundamental group T = (x, y \ x 3 = y 2 ) of the trefoil knot has a complete 
rewriting system with six logged rules shown in Table 5. 



rule 


label 


(Yy,\d) 


a 3 






(yY, id) 








(x 3 ,y 2 ) 


«5 






(y 2 x,xy 2 ) 


a 6 


= 


• (xa 5 ) 


(X, x 2 Y 2 ) 


dj 


= {Xal 1 


) • (Xyc^y) • (Xab^Y 2 ) • (aix 2 y 2 ) 


(Yx, yxY 2 ) 


a s 


= (Yxa^ 


x ) • (Yxya^Y) • (yro^y 2 ) • (Ya 5 xY 2 ) • (a 3 yxy 2 ) 



Table 5 : Logged rewrite rules for the trefoil monoid. 



The ordering used here is the wreath product order with X > x > Y > y. A 
group version of these logged rules is given in (Heyworth and Wensley, 2003). 

The non-deterministic automaton N T has 7 states, and there are 12 states in 
the determinized automaton, reducing to 7 states on minimization. The au- 
tomaton A r is pictured in Figure 2. (For clarity, the sink state and transitions 
to it have been excluded. All states are accepting, so double circles have been 
omitted.) 
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Figure 2 : Sketch of the word acceptor A r for T 
A regular expression for the language accepted by A r is 

(1 + y)x(yx + xyx)*{\ + x)(y* + F + ) + (y* + 

We consider subgroups H = (x) and K = (y). This is an example which 
apparently cannot be computed using algorithms previously available. The 
double coset rewriting system initially requires the additional rules 

[ Pi = (HX,H), fa = (Hx,H), fa = (YK, K), fa = (yK, K) }. 

The kan package includes a limited version of the Knuth-Bendix functions 
which stop the calculation after a specified number A of rules have been added 
to the system. Subsets of rules which involve either H , or K, or both may 
then be extracted. Adopting the wreath product order with K > H > X > 
x > Y > y we find that (3\ reduces, and there are no additional -fT-rules or 
HK-rvAes. As A increases we obtain an increasing number of rules from the 
following infinite set: 

{ (Hwy 2 , Hw), (HwY,Hwy) w is any word in {yx,yx 2 }* }. 

Note that the words yx and yx 2 label directed cycles in Figure 2. It is straight- 
forward to verify that, if we add all these rules, the system is complete, despite 
the fact that it is infinite: the H acting as a tag on the left restricts the possible 
overlaps severely and only a few cases need be checked. 

Also as A increases, the left-hand sides of the rules may be used to form the 
three trees in the double coset automaton of Figure 1. Applying the construc- 
tion of Theorem 6, we obtain a sequence of minimized automata {A A } which 
rapidly converges to the automaton depicted in Figure 3. Indeed, with A = 10, 
we obtain a sufficient set of three if -rules and two -fT-rules, namely {fa, 03, A} 
together with 



rule 


label 


P 5 = (Hy 2 ,H) 
(3 6 = (HY,Hy) 


Ha^ 1 • (3 2 x 2 • (3 2 x • (3 2 
/3 5 _1 y • Hya 4 



1(3 



Figure 3 



© 

Double coset automaton for the trefoil group. 



The normal forms can be read straight off the automaton: 

{ HwK w is any word in {yx,yx 2 }* }. 

For an example of logged reduction, consider the double coset HFK, where Y 
is a normal form for A r . Applying H(3^ we obtain immediately that HYK — > 
H K. Alternatively, we may apply 

/3 6 K»Hf3 4 = fi^YK • ^ l xYK • 0^x 2 YK • Ha 5 K • Hya 4 K • H0 4 , 

which gives successive rewrites 

HYK -► HxYK -» Hx 2 YK -» Hx 3 YK -> Hy 2 YK -» FyET -» 

Applying equation (1) to these two reductions (where w\ = Y and 1^2 = id) gives 

^K)^(F)- 1 = y" 1 !/ = 1 G = 0(w 2 ), 
^(x)) 3 ^^)^^)- 1 = ^(a; 3 r 2 ) = 1 G = 0(w 2 ), 

and so we have obtained an endorewrite HP^ 1 • (3 6 K • Hfii E E(HK). 

Example 9 (A group with an infinite rewriting system) 

When the group G has an infinite rewriting system the double coset rewriting 
system will also be infinite. In this case it may be possible to use the package 
KB MAG to compute a word acceptor for G. In the kan package the finite state 
automaton provided by KB MAG is converted to a deterministic automaton 
in the format used by the automata package, and then included as the N. G 
part of the double coset automaton shown in Figure 1. It is still necessary 
to find appropriate sets of rules Rh,Rk and i?HK and, since Rq is infinite, 
the limited Knuth-Bendix functions should again be used. An interactive use 
of the package is required: experimenting with different limits gives partial 
results from which we may be able to deduce exact answers. 

In the following example we take G to be the group with generators {a, b} and 
relators [a 3 , b 3 , (ab) 3 ]. The normal forms of the monoid elements are strings 



17 



alternating in a or A with b or B. Not all such strings are irreducible, for 
example bob — > ABA and abaB — > BAb. The automatic structure computed 
by KB MAG has a word acceptor with 17 states. 

If we take H to be generated by [ab] and K to be generated by [ba], we find 
that all three additional sets of rules are infinite: 



The sequence {A A } of minimized automata exhibits an increasing number of 
states, reaching 48 at A = 200. On inspection, we find that these automata 
have a common set of states (the right-hand half in Figure 4), and two chains 
which gradually increase in length. Taking A = liniA^oo A A , these chains shrink 
to two-state loops, and we obtain the 19 state automaton shown in Figure 4, 
where the norm and sink states have been omitted, and states shown with a 
double circle are those having a iT-transition to norm. 



The language recognised by A is 

H ( a + (id + A)(bA)* + AB(aB)*A + b(aB) + A(bA)* + A(bA)*(Ba)*b ) K. 

7 Induced actions and left Kan extensions 

The algorithms we have presented arose as part of a programme of applying 
categorical constructions in computer algebra so as to allow increased flexibil- 
ity and a wider range of applications. It is worth recalling (Mac Lane, 1998) 
in noting that our goal is not the reduction of the familiar to the unfamiliar, 
but the extension of the familiar to cover many more cases. In this section, 
we shall demonstrate this by showing how the double coset problem is an in- 
stance of the much more general construction of induced actions of monoids 
and categories. Our aim also is to advertise this construction, which has many 



{ (Hab,H), (HaB, Hb), (H(bA) n B, H(Ab) n a),n ^ } 
{ (baK, K), (BaK, bK), (B (Ab) n K , a(bA) n K) , n > } , 
{ (Hb(Ab) n K,H(Ab) n AK),n^ }. 




Figure 4 : Automaton A for Example 9. 
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uses apart from those given here (see, for example, (Brown and Heyworth, 
2000)). 

Let F : M — > N be a morphism of monoids, and let M, N act on sets X, Y 
respectively. These are right actions, and we use the notation x m , y n . An F- 
morphism e : X — > Y is a function of sets such that e(x' m ) = (ex) F ^ m - ) for all 
m G M, x G X. In the case F = In we call e an N-morphism. The F- morphism 
e is said to be universal if, for any action of N on a set Z and any F-morphism 
(f) : X — > Z, there is a unique N-morphism ip : Y - Z such that o e = <ft. 
When e is universal we say that the action of N on Y is induced from X by 
F, and we write Y = F*(X). 

It is easy to construct this Y from the action IxM-*I and the morphism 
F. We let Y' = X x N with N-action (x, n) n := (x,nn'), and define Y" to be 
Y' factored by the equivalence relation generated by 

(x m ,n) « (x, F(m)n), n G N, m G M, x G X, (2) 

with e : X — > Y mapping x to the equivalence class of (x, In). 

A common example is where M is a subgroup of a group N, F is the inclusion 
morphism, and X is a singleton. Then F*(X) can easily be identified with 
M\N, the set of left cosets of M in N, and the usual N-action. Note also that 
we immediately have an extension of the problem of determining cosets to 
that of extending an action of the subgroup M on a set X to an action of N 
on a new set F*(X). 

This notion of induced action is easily extended to the case where M, N are 
small categories, F : M — >• N is a functor, and the action of M is given by a 
functor X : M — ► Sets (see below). In (Heyworth, 1999; Brown and Heyworth, 
2000) string rewriting for a monoid presentation of a monoid N was generalised 
to string rewriting for an induced action of categories, given a presentation of 
the data for this. Our string rewriting procedure for double cosets is a special 
case of this general form of string rewriting for induced actions of categories. 

Note that induced actions are also well known in category theory as left Kan 
extensions, and have many applications under that terminology. In that set- 
ting, it is often convenient to describe a choice of left Kan extensions for all 
actions as giving a functor of functor categories 

F* : (Sets) M -> (Sets) N 

which is left adjoint to the standard functor 

F* : (Sets) N -> (Sets) M 

given by composition with F. An implication of this is that F* preserves 
colimits of actions, but we do not pursue this theme. 
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The construction of the functor Y = F*(X) : N — > Sets in the category case 
is an easy generalisation of that for monoids, but with appropriate attention 
to the objects of the categories concerned. For b G Ob(N) let Y'(b) be the 
disjoint union of the sets X(c) x U(F(c),b) for all c G Ob(M). On Y'(b) we 
impose the equivalence relation generated by equation (2) to give Y(b) as the 
quotient. Note that now x G X(c), x m G X(c'), m G M(c, c') and n G N(c', b), 
as in Figure 5. The action of N is induced by (x, n) n = (x, nn') as before. This 
construction is known in category theory as that of a coend. 




Figure 5 : Action induced from X by F . 



We now apply this construction to the double coset problem. Let T be a 
set with commuting right H- and K-actions, so that ( i y h ) k = ( 7 fc ) h for all 
7 G T, h G H,k G K. Intuitively we prefer to think of the H-action as a left 
action, defining h 7 := j h . We define a category M = H e> K with objects 
{1, 2} and the following elements: 

M(1,1) = H, M(2,2) = K, M(l, 2) = H x K, M(2, 1) = 0. 

Composition in M is given by the usual multiplication in H, K and by 

h 2 (h 1 ,ki)k2 = (h2hi,kik 2 ), so that (h, k) = h(l,l)k. 

For the M-action X we take X(l) = T x {1} and X(2) = T x {2} to be copies 
of T. For the morphisms we define 

X(/ i )( 7 ,l) = r 1 7,l), X(fc)(7,2) = ( 7 fc ,2), X(h, fc)( 7 , 1) = T V, 2), 
so 1) is the isomorphism between copies of T mapping (7, 1) to (7, 2). 

Proposition 10 // N zs i/ie trivial category with one object and 1 the only 
morphism, X is the M-action given above, and F is the unique functor M — > U, 
then F*(X) may be identified with the set of orbits H\r/K. 



20 



Proof An N-action Y is just a set Y(0) and the identity function Y(1q) on 
y(0). The construction above gives Y'(0) = (T x {1} x {1 }) U (r x {2} x {1 }). 
Applying the equivalence rule (2) with m = h G H, m = (1, 1) and m = e K, 
we obtain 

(* _1 7 , Mo) « ( 7 , Mo), (7, 2, 1 ) « (7, 1, lo), (7*, 2, 1„) « (7, 2, 1 ), 

identifying the two copies of T and modelling the H, K actions on 17 □ 

When T = G and the actions are g h := h _1 g, g k := gk, then .F*(X) may be 
identified with the set of double cosets H\G/K. 

To summarise, as with cosets and other problems in computational algebra, 
double cosets are an instance of the general problem of computing Kan ex- 
tensions or induced actions of categories. By developing string rewriting for 
computing such Kan extensions, we therefore have a generic algorithm appli- 
cable to all of these problems. 



8 Conclusions 

One of the nice outcomes of our results is that existing powerful string rewrit- 
ing software can immediately be applied to double coset problems, provided 
that we have a presentation for the group and generating sets for the sub- 
groups. Of course the algorithm may not terminate, but it will in all cases 
where there are a finite number of cosets Hg and gK, (where the existing enu- 
merative methods could be used), and also in some cases where there are an 
infinite number of double cosets and enumerative methods are likely to fail. 

We have developed procedures for logged string rewriting, which via a 2- 
categorical structure, records all computations made from the original pre- 
sentations. This enables us to compute not only whether two elements of 
a group lie within the same double coset but also, using logged rewriting, to 
produce a proof of this in the case that they do. We expect to release the GAP 
functions to do both determining and proving via logged string rewriting as a 
share package. 

In this paper we apply our results only to group theory, but as we showed 
above, they hold much more generally. It might be interesting to see whether 
there are analogues to the double coset problem in other structures such as 
monoids or algebras - structures where we also already have Kan extension 
rewriting methods available to us. 
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